9 research outputs found
Neuron Circuit Characterization in a Neuromorphic System
Spiking neural networks can solve complex tasks in an event-based processing strategy, inspired by the brain. One special kind of neuron model, the AdEx model, allows to reproduce several types of firing patterns, which have been found in biological neurons and may be of functional importance. In this thesis we characterize the analog neuron circuit implementation of this model within the full-custom
HICANN ASIC. As the central unit of the BrainScaleS accelerated neuromorphic computing platform, it provides a tool to emulate large neural networks in short time and helps to better understand the brain.
Characterization of the neuron circuits leads to calibration of each sub-circuit, translating the desired AdEx model parameters to their corresponding HICANN parameters for each individual neuron. Device mismatch in VLSI manufacturing
leads to expected variation from design parameters. These variations can be counteracted by adjustable parameters within the circuits. A wafer-scale BrainScaleS system contains over 1.9·10^5 neuron circuits with millions of parameters. Due to the large scale of the system, methods need to be fully automated in a robust way.
Characterizations presented in this work are performed from transistor level simulation to wafer-scale hardware measurements. Our commissioning and calibration efforts are enabling neural network experiments, including complex firing patterns that are computationally expensive when implemented in traditional numerical simulations
Accelerated physical emulation of Bayesian inference in spiking neural networks
The massively parallel nature of biological information processing plays an
important role for its superiority to human-engineered computing devices. In
particular, it may hold the key to overcoming the von Neumann bottleneck that
limits contemporary computer architectures. Physical-model neuromorphic devices
seek to replicate not only this inherent parallelism, but also aspects of its
microscopic dynamics in analog circuits emulating neurons and synapses.
However, these machines require network models that are not only adept at
solving particular tasks, but that can also cope with the inherent
imperfections of analog substrates. We present a spiking network model that
performs Bayesian inference through sampling on the BrainScaleS neuromorphic
platform, where we use it for generative and discriminative computations on
visual data. By illustrating its functionality on this platform, we implicitly
demonstrate its robustness to various substrate-specific distortive effects, as
well as its accelerated capability for computation. These results showcase the
advantages of brain-inspired physical computation and provide important
building blocks for large-scale neuromorphic applications.Comment: This preprint has been published 2019 November 14. Please cite as:
Kungl A. F. et al. (2019) Accelerated Physical Emulation of Bayesian
Inference in Spiking Neural Networks. Front. Neurosci. 13:1201. doi:
10.3389/fnins.2019.0120
Inference with Artificial Neural Networks on Analog Neuromorphic Hardware
The neuromorphic BrainScaleS-2 ASIC comprises mixed-signal neurons and
synapse circuits as well as two versatile digital microprocessors. Primarily
designed to emulate spiking neural networks, the system can also operate in a
vector-matrix multiplication and accumulation mode for artificial neural
networks. Analog multiplication is carried out in the synapse circuits, while
the results are accumulated on the neurons' membrane capacitors. Designed as an
analog, in-memory computing device, it promises high energy efficiency.
Fixed-pattern noise and trial-to-trial variations, however, require the
implemented networks to cope with a certain level of perturbations. Further
limitations are imposed by the digital resolution of the input values (5 bit),
matrix weights (6 bit) and resulting neuron activations (8 bit). In this paper,
we discuss BrainScaleS-2 as an analog inference accelerator and present
calibration as well as optimization strategies, highlighting the advantages of
training with hardware in the loop. Among other benchmarks, we classify the
MNIST handwritten digits dataset using a two-dimensional convolution and two
dense layers. We reach 98.0% test accuracy, closely matching the performance of
the same network evaluated in software
Neuromorphic Hardware In The Loop: Training a Deep Spiking Network on the BrainScaleS Wafer-Scale System
Emulating spiking neural networks on analog neuromorphic hardware offers
several advantages over simulating them on conventional computers, particularly
in terms of speed and energy consumption. However, this usually comes at the
cost of reduced control over the dynamics of the emulated networks. In this
paper, we demonstrate how iterative training of a hardware-emulated network can
compensate for anomalies induced by the analog substrate. We first convert a
deep neural network trained in software to a spiking network on the BrainScaleS
wafer-scale neuromorphic system, thereby enabling an acceleration factor of 10
000 compared to the biological time domain. This mapping is followed by the
in-the-loop training, where in each training step, the network activity is
first recorded in hardware and then used to compute the parameter updates in
software via backpropagation. An essential finding is that the parameter
updates do not have to be precise, but only need to approximately follow the
correct gradient, which simplifies the computation of updates. Using this
approach, after only several tens of iterations, the spiking network shows an
accuracy close to the ideal software-emulated prototype. The presented
techniques show that deep spiking networks emulated on analog neuromorphic
devices can attain good computational performance despite the inherent
variations of the analog substrate.Comment: 8 pages, 10 figures, submitted to IJCNN 201
Pattern representation and recognition with accelerated analog neuromorphic systems
Despite being originally inspired by the central nervous system, artificial
neural networks have diverged from their biological archetypes as they have
been remodeled to fit particular tasks. In this paper, we review several
possibilites to reverse map these architectures to biologically more realistic
spiking networks with the aim of emulating them on fast, low-power neuromorphic
hardware. Since many of these devices employ analog components, which cannot be
perfectly controlled, finding ways to compensate for the resulting effects
represents a key challenge. Here, we discuss three different strategies to
address this problem: the addition of auxiliary network components for
stabilizing activity, the utilization of inherently robust architectures and a
training method for hardware-emulated networks that functions without perfect
knowledge of the system's dynamics and parameters. For all three scenarios, we
corroborate our theoretical considerations with experimental results on
accelerated analog neuromorphic platforms.Comment: accepted at ISCAS 201
A Scalable Approach to Modeling on Accelerated Neuromorphic Hardware.
Neuromorphic systems open up opportunities to enlarge the explorative space for computational research. However, it is often challenging to unite efficiency and usability. This work presents the software aspects of this endeavor for the BrainScaleS-2 system, a hybrid accelerated neuromorphic hardware architecture based on physical modeling. We introduce key aspects of the BrainScaleS-2 Operating System: experiment workflow, API layering, software design, and platform operation. We present use cases to discuss and derive requirements for the software and showcase the implementation. The focus lies on novel system and software features such as multi-compartmental neurons, fast re-configuration for hardware-in-the-loop training, applications for the embedded processors, the non-spiking operation mode, interactive platform access, and sustainable hardware/software co-development. Finally, we discuss further developments in terms of hardware scale-up, system usability, and efficiency
Versatile emulation of spiking neural networks on an accelerated neuromorphic substrate
We present first experimental results on the novel BrainScaleS-2 neuromorphic
architecture based on an analog neuro-synaptic core and augmented by embedded
microprocessors for complex plasticity and experiment control. The high
acceleration factor of 1000 compared to biological dynamics enables the
execution of computationally expensive tasks, by allowing the fast emulation of
long-duration experiments or rapid iteration over many consecutive trials. The
flexibility of our architecture is demonstrated in a suite of five distinct
experiments, which emphasize different aspects of the BrainScaleS-2 system